Method and apparatus for detecting obstacle, electronic device and storage medium

ABSTRACT

A method and apparatus for detecting an obstacle, an electronic device, and a storage medium. A specific implementation of the method includes: detecting, by a millimeter-wave radar, position points of candidate obstacles in front of a vehicle; detecting, by a camera, a left road boundary line and a right road boundary line of a road on which a vehicle is located; separating the position points of the candidate obstacles according to the left road boundary line and the right road boundary line of the road on which the vehicle is located, and extracting position points between the left road boundary line and the right road boundary line; projecting the position points between the left road boundary line and the right road boundary line onto an image; and detecting, based on projection points of the position points on the image, a target obstacle in front of the vehicle.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese Patent Application No.202010166864.6, filed with the China National Intellectual PropertyAdministration (CNIPA) on Mar. 11, 2020, the contents of which areincorporated herein by reference in their entirety.

TECHNICAL FIELD

The present application relates to the technical field of intelligentvehicles, and further relates to an automatic driving technology, andmore particularly to a method and apparatus for detecting obstacle, anelectronic device, and a storage medium.

BACKGROUND

Environmental perception is the basis of intelligent driving technology.By installing sensing devices on to vehicle to sense the surroundingenvironment, intelligent auxiliary driving is realized. However, thevehicle sensing device has limited sensing range due to factors such asfixed position thereof or effective viewing angle thereof, andtherefore, the acquired sensed information cannot meet the requirementof intelligent driving. In particular, in the case of automatic driving,the requirement for comprehensive environmental information is higher inorder to ensure safe driving. In automatic driving schemes, radarsensors and visual sensors are two conventional sources of sensedinformation. Therefore, in the conventional environment sensing method,a radar ranging sensing module or a visual ranging sensing module isgenerally used to sense the environmental information around thevehicle.

In the radar ranging sensing module, a radio wave is transmitted by amillimeter-wave radar, and then a return wave is received, and positiondata of a target is measured based on the time difference between thetransmitting and receiving. The millimeter-wave radar has rangingreliability and long-range sensing capability. However, the radar doesnot have discrimination power in the height direction, and a radarreflection point such as a viaduct or a billboard is falsely-detected asa front obstacle, which easily produces a cadence braking or the like.In the visual ranging sensing module, the on-line camera calibrationtechnology is used, and change of the vehicle steering attitude ismonitored in real time, the monocular ranging error caused by the changeof the road condition is reduced. Camera calibration refers to that, inimage measurement processes and machine vision applications, in order todetermine the relationship between the three-dimensional geometricposition of a point on the surface of a space object and itscorresponding point in an image, a geometric model of camera imagingmust be established, these geometric model parameters being cameraparameters. In most conditions, these parameters can only be obtainedthrough experiments and calculations; and the process of solving theparameters (internal parameters, external parameters, distortionparameters) is called camera calibration. However, for an obstacle thatis not in the same plane as the vehicle, the precondition for the groundpoint monocular projection ranging is not satisfied, and therefore, theranging result is also inaccurate. However, the on-line cameracalibration technique can only correct the error caused by vehiclejitter, and cannot change the situation that the obstacle and thevehicle are not in the same plane.

SUMMARY

Embodiments of the present disclosure provide a method and apparatus fordetecting an obstacle, an electronic device and a storage medium, whichmay detect an obstacle in front of the vehicle more accurately, andsolves the problems of that the positioning of the obstacle in theheight direction is inaccurate while the monocular camera cannot achievean accurate ranging when the obstacle and the vehicle are not in thesame plane.

In a first aspect, some embodiments of the present disclosure provide amethod for detecting an obstacle, the method includes:

detecting, by a millimeter-wave radar, position points of candidateobstacles in front of a vehicle;

detecting, by a camera, a left road boundary line and a right roadboundary line of a road on which a vehicle is located;

separating the position points of the candidate obstacles according tothe left road boundary line and the right road boundary line of the roadon which the vehicle is located, and extracting position points betweenthe left road boundary line and the right road boundary line;

projecting the position points between the left road boundary line andthe right road boundary line onto an image; and

detecting, based on projection points of the position points on theimage, a target obstacle in front of the vehicle.

The above embodiments have following advantages or beneficial effect: inembodiments of the present disclosure, the position points of thecandidate obstacles can be separated by the left and right road boundarylines, and the position points between the left and right road boundarylines can be extracted; the position points between the left and rightroad boundary lines are projected onto an image; and a target obstaclein front of the vehicle is detected through the projection points of theposition points on the image, thereby achieving the purpose of detectingan obstacle in front of the vehicle more accurately. In the conventionalobstacle detection method, a millimeter-wave radar or a monocular camerais usually used separately to detect an obstacle, and themillimeter-wave radar has a problem of that the positioning of theobstacle in the height direction is inaccurate, while the monocularcamera has the problem of that an accurate ranging cannot be achievedwhen the obstacle and the vehicle are not in the same plane. Thesolution of the present disclosure adopts the technical means of mergingthe millimeter-wave radar and the camera, the problems in the prior artthat the positioning of the obstacle in the height direction isinaccurate while the monocular camera cannot achieve an accurate rangingwhen the obstacle and the vehicle are not in the same plane areovercome, and further, the technical effect of more accurately detectingan obstacle in front of the vehicle is achieved; Moreover, the technicalsolution of embodiments of the present disclosure is simple andconvenient to implement, convenient to popularize, and wider inapplication range.

In the above embodiments, the detecting, based on the projection pointsof the position points on the image, the target obstacle in front of thevehicle, includes:

acquiring, on the image, regions of interest (ROIs) corresponding to theprojection points;

calculating detection results of the ROIs corresponding to theprojection points through a depth convolution network, wherein adetection result comprises: a detection result indicting existence of anobstacle or a detection result indicting non-existence of an obstacle;and

detecting the target obstacle in front of the vehicle according to thedetection results of the ROIs corresponding to the projection points.

The above embodiment have following advantages or beneficial effect: theabove embodiment may calculate the detection results of the ROIscorresponding to the projection points through a depth convolutionnetwork, thus may accurately detects a target obstacle in front of thevehicle based on the detection results of the ROIs corresponding to theprojection points.

In the above embodiments, the calculating the detection results of theROIs corresponding to the projection points through the depthconvolution network, includes:

inputting ROIs corresponding to the projection points into the depthconvolution network;

acquiring, in the ROIs corresponding to the projection points,perception results of the position points;

calculating the detection results of the ROIs corresponding to theprojection points based on the perception results of the positionpoints; and

outputting, by the depth convolution network, the detection results ofthe ROIs corresponding to the projection points.

The above embodiment have following advantages or beneficial effect: theabove embodiment may calculate the detection results of the ROIscorresponding to the projection points based on the perception resultsof the position points. The perception result of each position point ineach ROI may be a measurement value measured by the millimeter-waveradar for the each position point, and the measurement value mayindicate whether an obstacle exists at the position point. Thus theelectronic device may accurately determine the detection result of eachROI based on the measurement values of the position points in the eachROI.

In the above embodiment, after the calculating the detection results ofthe ROIs corresponding to the projection points through the depthconvolution network, and before the detecting the target obstacle infront of the vehicle according to the detection results of the ROIscorresponding to the projection points, the method further includes:

extracting from the image, based on pre-determined position pointsfalsely-detected by the radar, projection points of the falsely-detectedposition points on the image;

filtering out, in the detection results of the ROIs corresponding to theprojection points outputted by the depth convolution network, theprojection points of the falsely-detected position points and detectionresults of ROIs corresponding to the projection points of thefalsely-detected position points; and

determining the obstacle in front of the vehicle based on detectionresults of ROIs corresponding to projection points after the filtration.

The above embodiment of the present disclose has following advantages orbeneficial effect: the above embodiment may filter out the projectionpoints of the falsely-detected position points and detection results ofROIs corresponding to the projection points of the falsely-detectedposition points, and then determine the obstacle in front of the vehiclebased on detection results of ROIs corresponding to projection pointsafter the filtration, so that it may determine the obstacle in front ofthe vehicle more accurately.

In the above embodiment, the detecting, based on the projection pointsof the position points on the image, the target obstacle in front of thevehicle, includes: detecting a type of the target obstacle according toprojection points of the position points on the image; and

calculating a distance between the target obstacle and the vehicle basedon the projection points of the position points on the image and apredetermined position point occupied by the vehicle.

The above embodiment has following advantages and beneficial effect:based on the projection points of the position points on the image andpredetermined position points occupied by the vehicle, the aboveembodiment may accurately detect the type of the target obstacle, andaccurately calculate a distance between the target obstacle and thevehicle.

In a second aspect, some embodiments of the present disclosure providean apparatus for detecting an obstacle, the apparatus includes: a radardetection module, a camera detection module, an extraction module, and afusion detection module.

the radar detection module is configured to detect, by a millimeter-waveradar, position points of candidate obstacles in front of a vehicle;

the camera detection module is configured to detect, by a camera, a leftroad boundary line and a right road boundary line of a road on which avehicle is located;

the extraction module is configured to separate the position points ofthe candidate obstacles according to the left road boundary line and theright road boundary line of the road on which the vehicle is located,and extract position points between the left road boundary line and theright road boundary line;

the fusion detection module is configured to project the position pointsbetween the left road boundary line and the right road boundary lineonto an image, and detect, based on projection points of the positionpoints on the image, a target obstacle in front of the vehicle.

In the above embodiment, the fusion detection module comprises anacquisition sub-module, a calculation sub-module and a detectionsub-module.

the acquisition sub-module is configured to acquire, on the image,regions of interest (ROIs) corresponding to the projection points;

the calculation sub-module is configured to calculate detection resultsof the ROIs corresponding to the projection points through a depthconvolution network, wherein a detection result comprises: a detectionresult indicting existence of an obstacle or a detection resultindicting non-existence of an obstacle; and

the detection sub-module is configured to detect the target obstacle infront of the vehicle according to the detection results of the ROIscorresponding to the projection points.

In the above embodiment, the calculation sub-module is furtherconfigured to input ROIs corresponding to the projection points into thedepth convolution network; acquire, in the ROIs corresponding to theprojection points, perception results of the position points; calculatethe detection results of the ROIs corresponding to the projection pointsbased on the perception results of the position points; and output, bythe depth convolution network, the detection results of the ROIscorresponding to the projection points.

In the above embodiment, the fusion detection module further comprises:a filtration sub-module for extracting from the image, based onpre-determined position points falsely-detected by the radar, projectionpoints of the falsely-detected position points on the image, andfiltering out, in the detection results of the ROIs corresponding to theprojection points outputted by the depth convolution network, theprojection points of the falsely-detected position points and detectionresults of ROIs corresponding to the projection points of thefalsely-detected position points; and

the detection sub-module is configured to determine the obstacle infront of the vehicle based on detection results of ROIs corresponding toprojection points after the filtration.

In the above embodiment, the fusion detection module is furtherconfigured to detect a type of the target obstacle according toprojection points of the position points on the image, and calculate adistance between the target obstacle and the vehicle based on theprojection points of the position points on the image and predeterminedposition points occupied by the vehicle.

In a third aspect, some embodiments of the present disclosure provide anelectronic device, includes:

one or more processors; and

a memory for storing one or more programs,

when the one or programs are executed by the one or more processors,cause the one or more processor to perform the method for detecting anobstacle according to any one of the embodiments of the presentdisclosure.

In a fourth aspect, some embodiments of the present disclosure provide astorage medium storing a computer program, when the computer program isexecuted by a processor, causes the method for detecting an obstacleaccording to any one of the embodiments of the present disclosure to beimplemented.

An embodiment of the present disclosure has following advantages andtechnical effects: the method and apparatus for detecting an obstacle,the electronic device and storage medium, first detects, by amillimeter-wave radar, position points of candidate obstacles in frontof a vehicle; then detects, by a camera, a left road boundary line and aright road boundary line of a road on which a vehicle is located; andseparates the position points of the candidate obstacles according tothe left road boundary line and the right road boundary line of the roadon which the vehicle is located, and extracts position points betweenthe left road boundary line and the right road boundary line; andfinally projects the position points between the left road boundary lineand the right road boundary line onto an image, and detects, based onprojection points of the position points on the image, a target obstaclein front of the vehicle. That is, in embodiments of the presentdisclosure, the position points of the candidate obstacles can beseparated by the left and right road boundary lines, and the positionpoints between the left and right road boundary lines can be extracted;the position points between the left and right road boundary lines areprojected onto an image; and a target obstacle in front of the vehicleis detected through the projection points of the position points on theimage, thereby achieving the purpose of detecting an obstacle in frontof the vehicle more accurately. In the conventional obstacle detectionmethod, a millimeter-wave radar or a monocular camera is usually usedseparately to detect an obstacle, and the millimeter-wave radar has aproblem of that the positioning of the obstacle in the height directionis inaccurate, while the monocular camera has the problem of that anaccurate ranging cannot be achieved when the obstacle and the vehicleare not in the same plane. The solution of the present disclosure adoptsthe technical means of merging the millimeter-wave radar and the camera,the problems in the prior art that the positioning of the obstacle inthe height direction is inaccurate while the monocular camera cannotachieve an accurate ranging when the obstacle and the vehicle are not inthe same plane are overcome, and further, the technical effect of moreaccurately detecting an obstacle in front of the vehicle is achieved;Moreover, the technical solution of embodiments of the presentdisclosure is simple and convenient to implement, convenient topopularize, and wider in application range.

The other effects of the above optional implementations will bedescribed below in conjunction with specific embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings are for providing a better understanding of the presentdisclosure and are not to be construed as limiting the scope of thepresent disclosure.

FIG. 1 is a schematic flow diagram of a method for detecting an obstacleaccording to Embodiment 1 of the present disclosure;

FIG. 2 is a schematic flow diagram of a method for detecting an obstacleaccording to Embodiment 2 of the present disclosure;

FIG. 3 is a schematic structural diagram of an apparatus for detectingan obstacle according to Embodiment 3 of the present disclosure;

FIG. 4 is a schematic structural diagram of a fusion detection moduleaccording to Embodiment 3 of the present disclosure;

FIG. 5 is a block diagram of an electronic device for implementing themethod for detecting an obstacle of embodiments of the presentdisclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

Exemplary embodiments of the present disclosure are described below withreference to the accompanying drawings, in which various details of theembodiments of the present disclosure are included to facilitateunderstanding, and are to be considered as exemplary only. Accordingly,one of ordinary skill in the art should recognize that various changesand modifications may be made to the embodiments described hereinwithout departing from the scope and spirit of the present disclosure.Also, for clarity and conciseness, descriptions of well-known functionsand structures are omitted from the following description.

Embodiment 1

FIG. 1 is a schematic flow diagram of a method for detecting an obstacleaccording to Embodiment 1 of the present disclosure, which may beperformed by an apparatus for detecting an obstacle or by an electronicdevice, the apparatus and the electronic device may be implemented insoftware and/or hardware, and the apparatus and the electronic devicemay be integrated in any intelligent device having a networkcommunication function. As shown in FIG. 1, the method for detecting anobstacle may include the following steps:

S101, detect, by a millimeter-wave radar, position points of candidateobstacles in front of a vehicle.

In a specific embodiment of the present disclosure, the electronicdevice may detect, by the millimeter-wave radar, position points of thecandidate obstacles in front of the vehicle. Specifically, duringdriving of the vehicle, the electronic device may transmit a radio wavethrough the millimeter-wave radar, then receive the return wave, andmeasure a coordinate of a position point of the candidate obstaclesaccording to the time difference between the transmitting and receiving,and the coordinates of these position points may be two-dimensionalcoordinates.

S102, detect, by a camera, a left road boundary line and a right roadboundary line of a road on which a vehicle is located.

In a specific embodiment of the present disclosure, the electronicdevice may detect, by a camera, the left and right road boundary linesof a road on which the vehicle is located. Specifically, during thedriving of the vehicle, the electronic device may photograph, by thecamera, the objects in front of the vehicle at fixed time intervals, andthen detect, in the photographed images, the left and right roadboundary lines of the road on which the vehicle is located.Specifically, if the left road boundary line cannot be detected in animage, the leftmost road edge of the road on which the vehicle islocated is taken as the left road boundary line of the road; if theright road boundary line cannot be detected in an image, the rightmostroad edge of the road on which the vehicle is located is taken as theright road boundary line of the road.

S103, separate the position points of the candidate obstacles accordingto the left road boundary line and the right road boundary line of theroad on which the vehicle is located, and extract position pointsbetween the left road boundary line and the right road boundary line.

In a specific embodiment of the present disclosure, the electronicdevice can separate the position points of the candidate obstacles basedon the left and right road boundary lines of the road on which thevehicle is located, and extract the position points between the left andright road boundary lines. Specifically, in the position points of thecandidate obstacles detected by the radar, following operations areperformed for each of the position points: if the coordinate of theposition point is between the left road boundary line and the right roadboundary line, or the coordinate of the position point is on the leftroad boundary line, or the coordinate of the position point is on theright road boundary line, then the position point is kept; if thecoordinate of the position point is outside the left road boundary line,or the coordinate of the position point is outside the right roadboundary line, the coordinate of the position point are deleted. By theabove-described operations for each of the position points, it canseparate the position points of the candidate obstacles and extract theposition points between the left and right road boundary lines.

S104, project the position points between the left road boundary lineand the right road boundary line onto an image; detect, based onprojection points of the position points on the image, a target obstaclein front of the vehicle.

In a specific embodiment of the present disclosure, the electronicdevice may project the respective location points between the left andright road boundary lines onto the image; and may detect a targetobstacle in front of the vehicle based on the projection points of theposition points on the image. Specifically, the electronic device maymark the ground points of the respective position points between theleft and the right road boundary lines on the image, and detect thetarget obstacle in front of the vehicle based on the ground points ofthe respective position points on the image. Specifically, theelectronic device may: acquire, on the image, regions of interest (ROIs)corresponding to the projection points; calculate detection results ofthe ROIs corresponding to the projection points through a depthconvolution network, where the detection result includes a detectionresult indicating an existence of an obstacle or a detection resultindicating non-existence of an obstacle; and then detect a targetobstacle in front of the vehicle according to the detection results ofthe ROIs corresponding to the projection points.

The method for detecting an obstacle according to embodiments of thepresent disclosure, first detects, by a millimeter-wave radar, positionpoints of candidate obstacles in front of a vehicle; then detects, by acamera, a left road boundary line and a right road boundary line of aroad on which a vehicle is located; and separates the position points ofthe candidate obstacles according to the left road boundary line and theright road boundary line of the road on which the vehicle is located,and extracts position points between the left road boundary line and theright road boundary line; and finally projects the position pointsbetween the left road boundary line and the right road boundary lineonto an image, and detects, based on projection points of the positionpoints on the image, a target obstacle in front of the vehicle. That is,in embodiments of the present disclosure, the position points of thecandidate obstacles can be separated by the left and right road boundarylines, and the position points between the left and right road boundarylines can be extracted; the position points between the left and rightroad boundary lines are projected onto an image; and a target obstaclein front of the vehicle is detected through the projection points of theposition points on the image, thereby achieving the purpose of detectingan obstacle in front of the vehicle more accurately.

In the conventional obstacle detection method, a millimeter-wave radaror a monocular camera is usually used separately to detect an obstacle,and the millimeter-wave radar has a problem of that the positioning ofthe obstacle in the height direction is inaccurate, while the monocularcamera has the problem of that an accurate ranging cannot be achievedwhen the obstacle and the vehicle are not in the same plane. Thesolution of the present disclosure adopts the technical means of mergingthe millimeter-wave radar and the camera, the problems in the prior artthat the positioning of the obstacle in the height direction isinaccurate while the monocular camera cannot achieve an accurate rangingwhen the obstacle and the vehicle are not in the same plane areovercome, and further, the technical effect of more accurately detectingan obstacle in front of the vehicle is achieved; Moreover, the technicalsolution of embodiments of the present disclosure is simple andconvenient to implement, convenient to popularize, and wider inapplication range.

Embodiment 2

FIG. 2 is a schematic flowchart of a method for detecting an obstacleaccording to Embodiment 2 of the present disclosure. As shown in FIG. 2,the method for detecting an obstacle may include the following steps:

S201, detect, by a millimeter-wave radar, position points of candidateobstacles in front of a vehicle.

In a specific embodiment of the present disclosure, the electronicdevice may detect, by the millimeter-wave radar, position points of thecandidate obstacles in front of the vehicle. Specifically, duringdriving of the vehicle, the electronic device may transmit radio wavesthrough the millimeter-wave radar, then receive return waves, andmeasure coordinates of the position points of the candidate obstaclesaccording to the time difference between the transmitting and receiving,and the coordinates of these position points may be two-dimensionalcoordinates.

S202, detect, by a camera, a left road boundary line and a right roadboundary line of a road on which a vehicle is located.

In a specific embodiment of the present disclosure, the electronicdevice may detect, by a camera, the left and right road boundary linesof a road on which the vehicle is located. Specifically, during thedriving of the vehicle, the electronic device may photograph, by thecamera, the objects in front of the vehicle at fixed time intervals, andthen detect, in the photographed images, the left and right roadboundary lines of the road on which the vehicle is located.Specifically, if the left road boundary line cannot be detected in animage, the leftmost road edge of the road on which the vehicle islocated is taken as the left road boundary line of the road; if theright road boundary line cannot be detected in an image, the rightmostroad edge of the road on which the vehicle is located is taken as theright road boundary line of the road.

S203, separate the position points of the candidate obstacles accordingto the left road boundary line and the right road boundary line of theroad on which the vehicle is located, and extract position pointsbetween the left road boundary line and the right road boundary line.

In a specific embodiment of the present disclosure, the electronicdevice can separate the position points of the candidate obstacles basedon the left and right road boundary lines of the road on which thevehicle is located, and extract the position points between the left andright road boundary lines. Specifically, in the position points of thecandidate obstacles detected by the radar, following operations areperformed for each of the position points: if the coordinate of theposition point is between the left road boundary line and the right roadboundary line, or the coordinate of the position point is on the leftroad boundary line, or the coordinate of the position point is on theright road boundary line, then the position point is kept; if thecoordinate of the position point is outside the left road boundary line,or the coordinate of the position point is outside the right roadboundary line, the coordinate of the position point are deleted. By theabove-described operations for each of the position points, it canseparate the position points of the candidate obstacles and extract theposition points between the left and right road boundary lines.

S204, project the position points between the left road boundary lineand the right road boundary line onto an image; and acquire, on theimage, regions of interest (ROIs) corresponding to the projectionpoints.

In a specific embodiment of the present disclosure, the electronicdevice may project respective position points between the left and rightroad boundary lines onto the image; and may acquire ROIs correspondingto the projection points on the image. Specifically, the ROIs of theprojection points may be regions of regular shapes centered on theposition points, for example, an ROI may be a circular region, arectangular region, or the like.

S205, calculate detection results of the ROIs corresponding to theprojection points through a depth convolution network, where a detectionresult comprises: a detection result indicting existence of an obstacleor a detection result indicting non-existence of an obstacle.

In a specific embodiment of the present disclosure, the electronicdevice may calculate the detection results of the ROIs corresponding tothe projection points through the depth convolution network, where adetection result comprises: a detection result indicting existence of anobstacle or a detection result indicting non-existence of an obstacle.Specifically, the electronic device may input an ROI corresponding tothe projection points into the depth convolution network; then mayacquire perception results of the position points in the ROIcorresponding to the projection points; then calculate the detectionresult of the ROI corresponding to the projection points based on theperception results of the position points; and then output, by the depthconvolution network, the detection result of the ROI corresponding tothe projection points. Specifically, the perception result of eachposition point in each ROI may be a measurement value measured by themillimeter-wave radar for the each position point, and the measurementvalue may indicate whether an obstacle exists at the position point. Inthis step, the electronic device may determine the detection result ofeach ROI based on the measurement values of the position points in theeach ROI. For example, assuming that an ROI region includes 50 positionpoints, in which 30 position points therein have measurement valuesindicating an existence of an obstacle, and 20 position points thereinhave measurement values indicating non-existence of an obstacle, thenthe electronic device may determine that the detection result of the ROIis that an obstacle exists.

S206, detect the target obstacle in front of the vehicle according tothe detection results of the ROIs corresponding to the projectionpoints.

In a specific embodiment of the present disclosure, the electronicdevice can detect the target obstacle in front of the vehicle based onthe detection results of the ROIs corresponding to the projectionpoints. Specifically, the electronic device may mark the detectionresults of the ROIs on the image. For example, if the detection resultof an ROI indicates an existence of an obstacle, the electronic devicemay mark, on the image, the ROI as 1. If the detection result of an ROIindicates non-existence of an obstacle, the electronic device may mark,on the image, the ROI as 0. In this way, the target obstacle in front ofthe vehicle can be detected based on the detection result of each ROI.

Preferably, in a specific embodiment of the present disclosure, afterthe electronic device calculates the detection results of the ROIscorresponding to the projection points through the depth convolutionnetwork, and before the target obstacle in front of the vehicle isdetected according to the detection results of the ROIs corresponding tothe projection points, the electronic device may further extract, fromthe image, projection points of falsely-detected position points on theimage based on the predetermined position points falsely-detected by theradar; then may filter out, in the detection results of the ROIscorresponding to the projection points outputted by the depthconvolution network, the projection points of the falsely-detectedposition points and detection result of ROI corresponding to theprojection points of the falsely-detected position points.

Preferably, in a specific embodiment of the present disclosure, theelectronic device can detect the type of the target obstacle accordingto the projection points of the respective position points on the image;and may calculate the distance between the target obstacle and thevehicle based on the projection points of position points on the imageand the predetermined position points occupied by the vehicle.

The method for detecting an obstacle according to embodiments of thepresent disclosure, first detects, by a millimeter-wave radar, positionpoints of candidate obstacles in front of a vehicle; then detects, by acamera, a left road boundary line and a right road boundary line of aroad on which a vehicle is located; and separates the position points ofthe candidate obstacles according to the left road boundary line and theright road boundary line of the road on which the vehicle is located,and extracts position points between the left road boundary line and theright road boundary line; and finally projects the position pointsbetween the left road boundary line and the right road boundary lineonto an image, and detects, based on projection points of the positionpoints on the image, a target obstacle in front of the vehicle. That is,in embodiments of the present disclosure, the position points of thecandidate obstacles can be separated by the left and right road boundarylines, and the position points between the left and right road boundarylines can be extracted; the position points between the left and rightroad boundary lines are projected onto an image; and a target obstaclein front of the vehicle is detected through the projection points of theposition points on the image, thereby achieving the purpose of detectingan obstacle in front of the vehicle more accurately. In the conventionalobstacle detection method, a millimeter-wave radar or a monocular camerais usually used separately to detect an obstacle, and themillimeter-wave radar has a problem of that the positioning of theobstacle in the height direction is inaccurate, while the monocularcamera has the problem of that an accurate ranging cannot be achievedwhen the obstacle and the vehicle are not in the same plane. Thesolution of the present disclosure adopts the technical means of mergingthe millimeter-wave radar and the camera, the problems in the prior artthat the positioning of the obstacle in the height direction isinaccurate while the monocular camera cannot achieve an accurate rangingwhen the obstacle and the vehicle are not in the same plane areovercome, and further, the technical effect of more accurately detectingan obstacle in front of the vehicle is achieved; Moreover, the technicalsolution of embodiments of the present disclosure is simple andconvenient to implement, convenient to popularize, and wider inapplication range.

Embodiment 3

FIG. 3 is a schematic structural diagram of an apparatus for detectingan obstacle according to Embodiment 3 of the present disclosure. Asshown in FIG. 3, the apparatus 300 includes a radar detection module301, a camera detection module 302, an extraction module 303, and afusion detection module 304.

The radar detection module 301 is configured to detect, by amillimeter-wave radar, position points of candidate obstacles in frontof a vehicle;

The camera detection module 302 is configured to detect, by a camera, aleft road boundary line and a right road boundary line of a road onwhich a vehicle is located;

The extraction module 303 is configured to separate the position pointsof the candidate obstacles according to the left road boundary line andthe right road boundary line of the road on which the vehicle islocated, and extract position points between the left road boundary lineand the right road boundary line;

The fusion detection module 304 is configured to project the positionpoints between the left road boundary line and the right road boundaryline onto an image, and detect, based on projection points of theposition points on the image, a target obstacle in front of the vehicle.

FIG. 4 is a schematic structural diagram of the fusion detection module304 provided in Embodiment 3 of the present disclosure. As shown in FIG.4, the fusion detection module 304 includes an acquisition sub-module3041, a calculation sub-module 3042, and a detection sub-module 3043.

The acquisition sub-module 3041 is configured to acquire, on the image,regions of interest (ROIs) corresponding to the projection points;

The calculation sub-module 3042 is configured to calculate detectionresults of the ROIs corresponding to the projection points through adepth convolution network, where a detection result includes: adetection result indicting existence of an obstacle or a detectionresult indicting non-existence of an obstacle;

The detection sub-module 3043 is configured to detect the targetobstacle in front of the vehicle according to the detection results ofthe ROIs corresponding to the projection points.

Further, the calculation sub-module 3042 is further configured to inputROIs corresponding to the projection points into the depth convolutionnetwork; acquire, in the ROIs corresponding to the projection points,perception results of the position points; calculate the detectionresults of the ROIs corresponding to the projection points based on theperception results of the position points; and output, by the depthconvolution network, the detection results of the ROIs corresponding tothe projection points.

Further, the fusion detection module further includes a filtrationsub-module 3044 (not shown in the figures) for extracting from theimage, based on pre-determined position points falsely-detected by theradar, projection points of the falsely-detected position points on theimage, and filtering out, in the detection results of the ROIscorresponding to the projection points outputted by the depthconvolution network, the projection points of the falsely-detectedposition points and detection results of ROIs corresponding to theprojection points of the falsely-detected position points;

The detection sub-module 3043 is configured to determine the obstacle infront of the vehicle based on detection results of ROIs corresponding toprojection points after the filtration.

Further, the fusion detection module 304 is specifically configured todetect a type of the target obstacle according to projection points ofthe position points on the image, and calculate a distance between thetarget obstacle and the vehicle based on the projection points of theposition points on the image and predetermined position points occupiedby the vehicle.

The apparatus for detecting an obstacle can perform the method providedin any one of the embodiments of the present disclosure, and hasfunction modules and beneficial effects corresponding to the executionmethod. The technical details not described in detail in the presentembodiment may be referred to the method for detecting an obstacleprovided in any embodiment of the present disclosure.

Embodiment 4

An embodiment of the present disclosure also provides an electronicdevice and a computer-readable storage medium.

FIG. 5 is a block diagram of an electronic device of the method fordetecting an obstacle according to an embodiment of the presentdisclosure. The electronic device is intended to represent various formsof digital computers, such as laptop computers, desktop computers,worktables, personal digital assistants, servers, blade servers,large-scale computers, and other suitable computers. The electronicdevice may also represent various forms of mobile devices, such aspersonal digital assistants, cellular telephones, smart phones, wearabledevices, and other similar computing devices. The components shownherein, their connections and relationships, and their functions are byway of example only and are not intended to limit the implementations ofthe present disclosure as described and/or claimed herein.

As shown in FIG. 5, the electronic device includes one or moreprocessors 501, a memory 502, and an interface for connectingcomponents, including a high-speed interface and a low-speed interface.The various components are connected to each other by different buses,and may be mounted on a common motherboard or installed in other ways asdesired. The processor may process instructions executed within theelectronic device, including instructions stored in or on a memory todisplay graphical information of the GUI on an external input/outputdevice, such as a display device coupled to the interface. In otherembodiments, multiple processors and/or multiple buses may be used withmultiple memories, if desired. Similarly, a plurality of electronicdevices may be connected, each providing a portion of necessaryoperations (e.g., as a server array, a set of blade servers, or amultiprocessor system). one processor 501 is taken as an example in FIG.5.

The memory 502 is a non-transitory computer readable storage mediumprovided herein. The memory stores instructions executable by at leastone processor, to cause the at least one processor to perform the methodfor detecting an obstacle provided herein. The non-transitorycomputer-readable storage medium of the present disclosure storescomputer instructions for causing a computer to perform the method fordetecting an obstacle provided herein.

As a non-transitory computer readable storage medium, the memory 502 maybe used to store non-transitory software programs, non-transitorycomputer executable programs, and modules, for example, the programinstructions/modules corresponding to the method for detecting anobstacle in embodiments of the present disclosure (for example, theradar detection module 301, the camera detection module 302, theextraction module 303, and the fusion detection module 304 shown in FIG.3). The processor 501 runs the software programs, instructions andmodules stored in the memory 502 to execute various functionalapplications and data processing of a server, that is, to implement themethod for detecting an obstacle of the above method embodiments.

The memory 502 may include a program storage area and a data storagearea. The program storage area may store an operating system and anapplication required for at least one function. The data storage areamay store data and the like created according to the usage of anelectronic device according to the method for detecting an obstacle. Inaddition, the memory 502 may include a high-speed random access memory,and may also include a non-transitory memory, e.g., at least one diskstorage device, a flash memory device or other non-volatile solid-statestorage devices. In some embodiments, the memory 502 may further includememories remotely arranged relative to the processor 501, where theremote memories may be connected to the electronic device of the methodfor detecting an obstacle by a network. An example of the above networkincludes but not limited to, the Internet, an enterprise intranet, alocal area network, a mobile communications network, and a combinationthereof.

The electronic device of the method for detecting an obstacle mayfurther include an input device 503 and an output device 504. Theprocessor 501, the memory 502, the input device 503, and the outputdevice 504 may be connected via a bus or in other modes. Connection by abus is used as an example in FIG. 5.

The input device 503 may be used for receiving entered digit orcharacter information, and generating a key signal input related to theuser setting and function control of the electronic device of the methodfor detecting an obstacle, such as a touch screen, a keypad, a mouse, atrack pad, a touch pad, a pointer bar, one or more mouse buttons, atrackball, a joystick, or the like. The output device 504 may include adisplay device, an auxiliary illumination device (e.g., an LED), atactile feedback device (e.g., a vibration motor), and the like. Thedisplay device may include, but is not limited to, a liquid crystaldisplay (LCD), a light emitting diode (LED) display, and a plasmadisplay. In some embodiments, the display device may be a touch screen.

The various embodiments of the systems and techniques described hereinmay be implemented in digital electronic circuit systems, integratedcircuit systems, application specific ASICs (application specificintegrated circuits), computer hardware, firmware, software, and/orcombinations thereof. These various embodiments may include: beingimplemented in one or more computer programs that may be executed and/orinterpreted on a programmable system including at least one programmableprocessor which may be a dedicated or general purpose programmableprocessor, it may receive data and instructions from a memory system, atleast one input device, and at least one output device, and transmit thedata and instructions to the memory system, the at least one inputdevice, and the at least one output device.

These computing programs (also referred to as programs, software,software applications, or code) include machine instructions of aprogrammable processor and may be implemented in high-level proceduresand/or object-oriented programming languages, and/or assembly/machinelanguages. As used herein, the terms “machine-readable medium” and“computer-readable medium” refer to any computer program product,device, and/or device (e.g., magnetic disk, optical disk, memory,programmable logic device (PLD)) for providing machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as machine-readable signals.The term “machine readable signal” refers to any signal used to providemachine instructions and/or data to a programmable processor.

To provide interaction with a user, the systems and techniques describedherein may be implemented on a computer having a display device (e.g., aCRT (cathode ray tube) or LCD (liquid crystal display) monitor) fordisplaying information to the user and a keyboard and a pointing device(e.g., a mouse or a trackball) through which a user can provide input tothe computer. Other types of devices may also be used to provideinteraction with a user. For example, the feedback provided to the usermay be any form of sensory feedback (e.g., visual feedback, auditoryfeedback, or tactile feedback). The input in any form, includingacoustic input, speech input, or tactile input, may be received from theuser.

The systems and techniques described herein may be implemented in acomputing system including a backend component (e.g., as a data server),or a computing system including a middleware component (e.g., anapplication server), or a computing system including a front-endcomponent (e.g., a personal computer having a graphical user interfaceor a web browser through which a user may interact with embodiments ofthe systems and techniques described herein), or a computing systemincluding any combination of such backend component, middlewarecomponent, or front-end component. The components of the system may beinterconnected by any form or medium of digital data communication(e.g., a communication network). Examples of communication networksinclude local region networks (LANs), wide region networks (WANs), theInternet, and block chain networks.

The computer system may include a client and a server. The client andserver are typically remote from each other and typically interactthrough a communication network. The relationship between the client andthe server is generated by a computer program running on thecorresponding computer and having a client-server relationship with eachother.

The technical solution of embodiments of the present disclosure, firstdetect, by a millimeter-wave radar, position points of candidateobstacles in front of a vehicle; then detects, by a camera, a left roadboundary line and a right road boundary line of a road on which avehicle is located; and separates the position points of the candidateobstacles according to the left road boundary line and the right roadboundary line of the road on which the vehicle is located, and extractsposition points between the left road boundary line and the right roadboundary line; and finally projects the position points between the leftroad boundary line and the right road boundary line onto an image, anddetects, based on projection points of the position points on the image,a target obstacle in front of the vehicle. That is, in embodiments ofthe present disclosure, the position points of the candidate obstaclescan be separated by the left and right road boundary lines, and theposition points between the left and right road boundary lines can beextracted; the position points between the left and right road boundarylines are projected onto an image; and a target obstacle in front of thevehicle is detected through the projection points of the position pointson the image, thereby achieving the purpose of detecting an obstacle infront of the vehicle more accurately. In the conventional obstacledetection method, a millimeter-wave radar or a monocular camera isusually used separately to detect an obstacle, and the millimeter-waveradar has a problem of that the positioning of the obstacle in theheight direction is inaccurate, while the monocular camera has theproblem of that an accurate ranging cannot be achieved when the obstacleand the vehicle are not in the same plane. The solution of the presentdisclosure adopts the technical means of merging the millimeter-waveradar and the camera, the problems in the prior art that the positioningof the obstacle in the height direction is inaccurate while themonocular camera cannot achieve an accurate ranging when the obstacleand the vehicle are not in the same plane are overcome, and further, thetechnical effect of more accurately detecting an obstacle in front ofthe vehicle is achieved; Moreover, the technical solution of embodimentsof the present disclosure is simple and convenient to implement,convenient to popularize, and wider in application range.

It should be understood that the various forms of processes may be used,and the steps may be reordered, added or deleted. For example, the stepsdescribed in the present disclosure may be performed in parallel orsequentially or in a different order, as long as the desired results ofthe technical solution disclosed in the present disclosure can berealized, and no limitation is imposed herein.

The foregoing detailed description is not intended to limit the scope ofthe present disclosure. It will be appreciated by those skilled in theart that various modifications, combinations, sub-combinations, andsubstitutions may be made depending on design requirements and otherfactors. Any modifications, equivalents, and improvements made withinthe spirit and principles of the present disclosure are to be includedwithin the scope of the present disclosure.

What is claimed is:
 1. A method for detecting an obstacle, comprising:detecting, by a millimeter-wave radar, position points of candidateobstacles in front of a vehicle; detecting, by a camera, a left roadboundary line and a right road boundary line of a road on which avehicle is located; separating the position points of the candidateobstacles according to the left road boundary line and the right roadboundary line of the road on which the vehicle is located, andextracting position points between the left road boundary line and theright road boundary line; projecting the position points between theleft road boundary line and the right road boundary line onto an image;acquiring, on the image, regions of interest (ROIs) corresponding to theprojection points; calculating detection results of the ROIscorresponding to the projection points through a depth convolutionnetwork, wherein a detection result comprises: a detection resultindicting existence of an obstacle or a detection result indictingnon-existence of an obstacle; extracting from the image, based onpre-determined position points falsely-detected by the radar, projectionpoints of the falsely-detected position points on the image; filteringout, in the detection results of the ROIs corresponding to theprojection points outputted by the depth convolution network, theprojection points of the falsely-detected position points and detectionresults of ROIs corresponding to the projection points of thefalsely-detected position points; determining the obstacle in front ofthe vehicle based on detection results of ROIs corresponding toprojection points after the filtration; and detecting the targetobstacle in front of the vehicle according to the detection results ofthe ROIs corresponding to the projection points.
 2. The method accordingto claim 1, wherein the calculating the detection results of the ROIscorresponding to the projection points through the depth convolutionnetwork comprises: inputting ROIs corresponding to the projection pointsinto the depth convolution network; acquiring, in the ROIs correspondingto the projection points, perception results of the position points;calculating the detection results of the ROIs corresponding to theprojection points based on the perception results of the positionpoints; and outputting, by the depth convolution network, the detectionresults of the ROIs corresponding to the projection points.
 3. Themethod according to claim 1, wherein the detecting, based on theprojection points of the position points on the image, the targetobstacle in front of the vehicle comprises: detecting a type of thetarget obstacle according to projection points of the position points onthe image; and calculating a distance between the target obstacle andthe vehicle based on the projection points of the position points on theimage and predetermined position points occupied by the vehicle.
 4. Anapparatus for detecting an obstacle, comprising: at least one processor;and a memory storing instructions, the instructions, when executed bythe at least one processor, cause the at least one processor to performoperations, the operations comprising: detecting, by a millimeter-waveradar, position points of candidate obstacles in front of a vehicle;detecting, by a camera, a left road boundary line and a right roadboundary line of a road on which a vehicle is located; separating theposition points of the candidate obstacles according to the left roadboundary line and the right road boundary line of the road on which thevehicle is located, and extracting position points between the left roadboundary line and the right road boundary line; and projecting theposition points between the left road boundary line and the right roadboundary line onto an image; acquiring, on the image, regions ofinterest (ROIs) corresponding to the projection points; calculatingdetection results of the ROIs corresponding to the projection pointsthrough a depth convolution network, wherein a detection resultcomprises: a detection result indicting existence of an obstacle or adetection result indicting non-existence of an obstacle; extracting fromthe image, based on pre-determined position points falsely-detected bythe radar, projection points of the falsely-detected position points onthe image; filtering out, in the detection results of the ROIscorresponding to the projection points outputted by the depthconvolution network, the projection points of the falsely-detectedposition points and detection results of ROIs corresponding to theprojection points of the falsely-detected position points; determiningthe obstacle in front of the vehicle based on detection results of ROIscorresponding to projection points after the filtration; and detectingthe target obstacle in front of the vehicle according to the detectionresults of the ROIs corresponding to the projection points.
 5. Theapparatus according to claim 4, wherein the calculating the detectionresults of the ROIs corresponding to the projection points through thedepth convolution network comprises: inputting ROIs corresponding to theprojection points into the depth convolution network; acquiring, in theROIs corresponding to the projection points, perception results of theposition points; calculating the detection results of the ROIscorresponding to the projection points based on the perception resultsof the position points; and outputting, by the depth convolutionnetwork, the detection results of the ROIs corresponding to theprojection points.
 6. The apparatus according to claim 4, wherein thedetecting, based on the projection points of the position points on theimage, the target obstacle in front of the vehicle comprises: detectinga type of the target obstacle according to projection points of theposition points on the image; and calculating a distance between thetarget obstacle and the vehicle based on the projection points of theposition points on the image and predetermined position points occupiedby the vehicle.
 7. A non-transitory computer-readable storage mediumstoring computer instructions, wherein the computer instructions, whenexecuted by a processor, cause the processor to perform operations, theoperations comprising: detecting, by a millimeter-wave radar, positionpoints of candidate obstacles in front of a vehicle; detecting, by acamera, a left road boundary line and a right road boundary line of aroad on which a vehicle is located; separating the position points ofthe candidate obstacles according to the left road boundary line and theright road boundary line of the road on which the vehicle is located,and extracting position points between the left road boundary line andthe right road boundary line; projecting the position points between theleft road boundary line and the right road boundary line onto an image;acquiring, on the image, regions of interest (ROIs) corresponding to theprojection points; calculating detection results of the ROIscorresponding to the projection points through a depth convolutionnetwork, wherein a detection result comprises: a detection resultindicting existence of an obstacle or a detection result indictingnon-existence of an obstacle; extracting from the image, based onpre-determined position points falsely-detected by the radar, projectionpoints of the falsely-detected position points on the image; filteringout, in the detection results of the ROIs corresponding to theprojection points outputted by the depth convolution network, theprojection points of the falsely-detected position points and detectionresults of ROIs corresponding to the projection points of thefalsely-detected position points; determining the obstacle in front ofthe vehicle based on detection results of ROIs corresponding toprojection points after the filtration; and detecting the targetobstacle in front of the vehicle according to the detection results ofthe ROIs corresponding to the projection points.
 8. The medium accordingto claim 7, wherein the calculating the detection results of the ROIscorresponding to the projection points through the depth convolutionnetwork comprises: inputting ROIs corresponding to the projection pointsinto the depth convolution network; acquiring, in the ROIs correspondingto the projection points, perception results of the position points;calculating the detection results of the ROIs corresponding to theprojection points based on the perception results of the positionpoints; and outputting, by the depth convolution network, the detectionresults of the ROIs corresponding to the projection points.
 9. Themedium according to claim 7, wherein the detecting, based on theprojection points of the position points on the image, the targetobstacle in front of the vehicle comprises: detecting a type of thetarget obstacle according to projection points of the position points onthe image; and calculating a distance between the target obstacle andthe vehicle based on the projection points of the position points on theimage and predetermined position points occupied by the vehicle.